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Abstract 

Background: Spinal cord injury (SCI) can leave the affected individuals with paraparesis or paraplegia, thus rendering 
them unable to ambulate. Since there are currently no restorative treatments for this population, novel approaches 
such as brain-controlled prostheses have been sought. Our recent studies show that a brain-computer interface (BCI) 
can be used to control ambulation within a virtual reality environment (VRE), suggesting that a BCI-controlled lower 
extremity prosthesis for ambulation may be feasible. However, the operability of our BCI has not yet been tested in a 
SCI population. 

Methods: Five participants with paraplegia or tetraplegia due to SCI underwent a 1 0-min training session in which 
they alternated between kinesthetic motor imagery (KMI) of idling and walking while their electroencephalogram 
(EEG) were recorded. Participants then performed a goal-oriented online task, where they utilized KMI to control the 
linear ambulation of an avatar while making 10 sequential stops at designated points within the VRE. Multiple online 
trials were performed in a single day, and this procedure was repeated across 5 experimental days. 

Results: Classification accuracy of idling and walking was estimated offline and ranged from 60.5% (p = 0.01 76) to 
92.3% (p = 1 .36 x 1 0 -20 ) across participants and days. Offline analysis revealed that the activation of mid-frontal areas 
mostly in the /x and low /3 bands was the most consistent feature for differentiating between idling and walking KMI. 
In the online task, participants achieved an average performance of 7.4 ± 2.3 successful stops in 273 ± 51 sec. These 
performances were purposeful, i.e. significantly different from the random walk Monte Carlo simulations (p < 0.01), 
and all but one participant achieved purposeful control within the first day of the experiments. Finally, all participants 
were able to maintain purposeful control throughout the study, and their online performances improved over time. 

Conclusions: The results of this study demonstrate that SCI participants can purposefully operate a self-paced BCI 
walking simulator to complete a goal-oriented ambulation task. The operation of the proposed BCI system requires 
short training, is intuitive, and robust against participant-to-participant and day-to-day neurophysiological variations. 
These findings indicate that BCI-controlled lower extremity prostheses for gait rehabilitation or restoration after SCI 
may be feasible in the future. 

Keywords: Spinal cord injury, Brain computer interface, Virtual reality environment, Electroencephalogram, 
Kinesthetic motor imagery, Gait, Ambulation, Locomotion 
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Introduction 

Spinal cord injury (SCI) can leave the affected individ- 
uals with severe paraparesis or paraplegia, thus render- 
ing them unable to ambulate. Since there are currently 
no restorative treatments for this population, technolog- 
ical approaches have been sought to substitute for the 
lost motor functions. Examples include robotic exoskele- 
tons [1], functional electrical stimulation (FES) systems 
[2], and spinal cord stimulators [3]. However, these sys- 
tems lack intuitive able-body-like supraspinal control, so 
the ambulation function of these devices is controlled 
manually. As a result, they monopolize the upper extrem- 
ities, which interferes with the users hand function. 
Due to these and other limitations, wheelchairs remain 
the primary means of mobility after SCI. Unfortunately, 
the extended reliance on wheelchairs typically lead to 
a wide variety of comorbidities that constitute the bulk 
of chronic SCI-related medical care costs [4-7]. Con- 
sequently, to address the above issues associated with 
the treatment of severe paraparesis and paraplegia after 
SCI, novel brain-controlled prostheses are currently being 
pursued [8,9]. 

Recent results by our group [8,9] suggest that an elec- 
troencephalogram (EEG) based brain-computer interface 
(BCI) controlled lower extremity prosthesis may be feasi- 
ble. More specifically, these studies demonstrated the suc- 
cessful implementation of a BCI system that controls the 
ambulation of an avatar (a stand-in for a lower extremity 
prosthesis) within a virtual reality environment (VRE). By 
using a data-driven machine learning approach to decode 
the users' kinesthetic motor imageries (KMIs), this BCI- 
controlled walking simulator enabled a small group of par- 
ticipants (one with paraplegia due to SCI [9]) to achieve 
intuitive and purposeful BCI control after a short training 
session. While the single SCI participant outperformed 
most able-bodied participants in this study, the operability 
of this system has not yet been tested in a SCI population. 
The successful implementation of the BCI-controlled 
walking simulator in a population of participants with SCI 
will establish the feasibility of future BCI-controlled lower 
extremity prostheses and will represent an important step 
toward developing novel approaches to gait rehabilitation 
after SCI. 

Extending the application of the BCI-controlled walk- 
ing simulator to a SCI population is faced with several 
problems. First, cortical reorganization, which is common 
after SCI [10-13], may cause the cortical representation 
of walking KMI to vary vastly from one SCI participant 
to another. Second, this representation may dramatically 
evolve over time when SCI participants are engaged in 
KMI training [14]. Finally, participants with SCI may 
interpret walking KMI either as motor imagery or as 
attempted walking, which in turn may result in multi- 
ple patterns of cortical activation across these individuals. 



Therefore, intuitive BCI operation under these conditions 
requires a system that can accommodate for the varia- 
tions of brain physiology across SCI individuals, time, and 
strategies. To address these problems, we used a data- 
driven machine learning method to decode walking KMIs 
in a small population of SCI individuals. This approach 
enabled 5 participants to achieve intuitive and self-paced 
operation of the BCI-controlled walking simulator after 
only minimal training. Furthermore, they were able to 
maintain this level of control over the course of several 
weeks. 

Methods 

Overview 

The goal of this study was to determine if individu- 
als with complete motor SCI can use intuitive con- 
trol strategies to purposefully operate a BCI-controlled 
walking simulator. To achieve this goal, 5 participants 
with SCI underwent a short training procedure where 
they performed alternating epochs of idling and walk- 
ing KMI while their EEG were recorded. These training 
EEG data were then analyzed to build decoding mod- 
els for subsequent online BCI operation. To ascertain 
purposeful BCI control, participants then performed 5 
sessions of an online BCI goal-oriented virtual walk- 
ing task [9]. This entire procedure was performed 5 
times over the course of several weeks to determine 
if participants' performances improved with additional 
practice. 

Participant recruitment 

This study was approved by the University of Califor- 
nia, Irvine Institutional Review Board. Four participants 
with paraplegia and one with tetraplegia due to SCI were 
recruited via physician referral from the Long Beach Vet- 
erans Affairs Spinal Cord Injury Center and other SCI 
outreach programs. The participants (see Table 1) gave 
their informed consent to participate in the study. Note 
that all participants were BCI naive and most of them per- 
formed the experimental procedures at a rate of once per 
week. 



Table 1 Demographic data 


Participant 


Gender 


Age 


SCI status 


1 


M 


34 


T1 1, ASIA A, 8 yr. post injury 


2 


M 


46 


T1, ASIA B,4yr. post injury 


3 


M 


43 


C5, Syringomyelia, 1 4 yr. post onset 


4 


M 


59 


T1 , ASIA B, 2 yr. post injury 


5 


M 


21 


T1 1, ASIA B, 1 yr. post injury 



List of participants with demographic data and level of SCI. ASIA = American 
Spinal Injury Association impairment scale. 
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Data acquisition 

Each participant was positioned ~0.8 to 1 m from 
a computer screen that displayed textual cues or the 
VRE. EEG were recorded using a 63-channel cap (Medi 
Factory, Heerlen, The Netherlands) with Ag-AgCl elec- 
trodes arranged according to the extended 10-20 Inter- 
national Standard. Conductive gel (Compumedics USA, 
Charlotte, NC) was applied to all electrodes and the 
30-Hz impedances between each electrode and the refer- 
ence electrode were maintained at < 10 KQ by abrading 
the scalp with a blunt needle. The EEG signals were 
amplified and digitized (sampling rate: 256 Hz, resolu- 
tion: 22 bits) using two linked NeXus-32 EEG bioampli- 
fiers (MindMedia, Roermond-Herten, The Netherlands). 
These signals were then streamed in real-time to a 
computer, re-referenced in a common average mode, 
band-pass filtered (0.01-40 Hz), and subsequently ana- 
lyzed. The above procedures were performed using 
custom-written MATLAB (MathWorks, Natick, MA) 
programs. 

Training procedure 

To facilitate intuitive control of ambulation within in a 
VRE, a EEG decoding model was developed that differ- 
entiates between idling and walking KMI. To this end, 
training EEG data were acquired while participants under- 
went 30-sec alternating epochs of idling and walking KMI 
over a 10-min session. Participants were instructed to per- 
form walking KMI and idling via automated textual cues 
displayed on the screen. For the walking KMI task, par- 
ticipants were instructed to vividly imagine themselves 
walking or to attempt to perform the cyclic leg movements 
of walking (note that these attempted movements did not 
produce any motor output). During this entire procedure, 
a computer labeling signal (to identify idling and walking 
KMI epochs) was recorded by an auxiliary data acquisi- 
tion system (MP150, Biopac Systems, Goleta, CA), and 
a common pulse train sent to both EEG and auxiliary 
data systems was used to synchronize the EEG and label- 
ing signals. Also, the participants were asked to remain 
still during the entire procedure, and their movement was 
monitored by the experimenter. If consistent movements 
were observed, the participant was asked to repeat the 
entire training procedure without making the undesired 
movements. 

Decoding model generation 

Offline signal analysis to generate EEG decoding models 
was performed in a manner similar to Wang et al. [9]. 
Briefly, EEG and labeling signals were first aligned and 
merged using the synchronization signal, and an itera- 
tive artifact rejection algorithm [15] was used to exclude 
EEG channels with excessive electromyogram (EMG) arti- 
facts from further analysis. This was done by removing 



those channels whose EEG amplitude exceeded an outlier 
voltage threshold. The EEG data from the remaining 
channels were then split into ~30 sec segments of idling 
and walking KMI states using the labeling signal. Each 
EEG segment was then divided into 5 randomly spaced, 
non-overlapping 4-sec trials for a total of 100 trials, which 
were then transformed into the frequency domain using 
the Fast Fourier Transform (FFT). Note that the remain- 
ing data in each segment were discarded. Finally, the 
power spectral density of each trial was integrated in 
2-Hz bins, resulting in 20 binned power spectral values 
per channel. 

Once the EEG segments were transformed into the fre- 
quency domain, a systematic search was performed to find 
the best contiguous frequency range for classification [9]. 
The dimension of the input data was then reduced using 
classwise principal component analysis (CPCA) [16,17], 
and discriminating features were extracted using either 
Fishers linear discriminant analysis (LDA) [18] or approx- 
imate information discriminant analysis (AID A) [19,20]. 
This resulted in the extraction of one-dimensional (ID) 
spatio-spectral features: 



/ = TOc(d) 



(1) 



where d e R BxC is a single-trial of EEG data with B fre- 
quency bins per channel and C retained EEG channels, 
<$>C : M. BxC — >► M. m is a piecewise linear mapping from 
the data space to an m-dimensional CPCA-subspace, and 
T : R m -> K is a LDA or AIDA transformation matrix. 

Once the single-trial data were reduced to ID spatio- 
spectral features, a linear Bayesian classifier: 



r e 



1, if P(l \f) > P(W |/*) 
W, otherwise 



(2) 



was designed in the feature domain, where P(X |/*) and 
P(W |/*) are the posterior probabilities of idling and 
walking classes given the observed feature, /*, respec- 
tively. Eq. (2) can be read as: "classify f* as idle class 
if the posterior probability of idling is greater than 
the posterior probability of walking given the feature, 
/*, and vice versa." The offline performance of the 
Bayesian classifier (2), expressed as a classification accu- 
racy, was then assessed by performing 10 runs of stratified 
10-fold cross-validation (CV) [21]. To determine the opti- 
mal parameters for classification, the lower bound of the 
frequency range was then increased in 2-Hz steps, and this 
procedure was repeated until the classifier performance 
stopped improving, allowing the optimal lower frequency 
bound to be determined. The upper bound of the optimal 
frequency range was determined in a similar manner. The 
classification accuracy was also used to decide whether 
to use AIDA or LDA as a feature extraction technique. 
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Figure 1 Finite State Machine. The BCI system as a binary state 
machine with walking and idling states represented by circles. The 
state transitions are represented by arrows, with transitions triggered 
by the conditions shown next to the arrows. Self-pointing arrows 
denote that the system remains in the present state. 



Finally, the parameters of the decoding model, includ- 
ing the optimal frequency range, the feature extraction 
mapping, and the classifier parameters, were saved for 
real-time EEG analysis. 

Online signal analysis 

During online operation, 0.5-sec-long blocks of EEG data 
were acquired in real time (at a rate of two blocks per 
second). This rate was limited by the computer processing 
speed and was empirically found to ensure data acqui- 
sition without dropping packets [9]. Of these 0.5 sec 
blocks, 0.75-sec-long segments (i.e. the most recent 0.5 
sec block plus 50% of the previous block) were assem- 
bled and processed using the methods described in the 
Decoding model generation section. Briefly, the EEG sig- 
nals were first band-pass filtered, then transformed into 
the frequency domain using FFT, and the power spectral 
densities over the optimal frequency range were inte- 
grated into 2-Hz bins. These spectral data were used as 
an input for the feature extraction algorithm (1). Finally, 





■ 




1 


1 




i 





Figure 2 Avatar within the VRE. The avatar controlled by the BCI 
within the VRE. The simulator is operated in 3 rd person view. The 
participant uses walking KMI to move the avatar to each NPC and 
idling KMI to dwell there for at least 2 sec. 



using Bayes' rule, the posterior probabilities of idling and 
walking classes given the observed EEG feature,/*, were 
calculated. 

Online calibration 

Since the BCI system is a binary state machine with 
an idling and a walking state, a brief calibration pro- 
cedure was required to identify the posterior proba- 
bility thresholds to transition between these two states 

Table 2 Offline performances 



Offline performances 



Participant 


Day 


P(correct |/*) (%) 


p-value 




1 


1 


71.9 ± 2.2 


6.29 x 


o- 


-6 




2 


89.4 ± 1 .2 


1.53 x 


0" 






3 


83.9 ± 2.0 


1.30 x 


0" 


-1 2 




4 


84.0 ± 1 .9 


1.30 x 


0" 


-1 2 




5 


82.2 ± 1.7 


6.55 x 


0" 


-1 2 




Avg. 


82.3 ± 6.4 


1.26 x 


0" 


-6 


2 


1 


62.2 ± 1.8 


6.00 x 


0" 


_3 




2 


62.0 ± 1.8 


1.05 x 


0" 


_2 




3 


60.5 ± 2.0 


1.76 x 


0" 


_2 




4 


91.6 ± 1.7 


1.60 x 


0" 


-1 9 




5 


82.5 ± 1.6 


6.55 x 


0" 


-1 2 




Avg. 


71.8 ± 14.3 


6.82 x 


0" 


-3 


3 


1 


90.3 ± 1.3 


1.66 x 


0" 


-18 




2 


83.9 ± 1.1 


1.30 x 


o- 


-12 




3 


72.8 ± 2.9 


2.35 x 


0" 


-6 




4 


81.0 ±2.1 


1.35 x 


0" 


-10 




5 


83.3 ± 1.2 


1.30 x 


0" 


-12 




Avg. 


82.3 ± 6.3 


4.69 x 


0" 


-7 


4 


1 


74.7 ± 1.7 


2.82 x 


0" 


-7 




2 


92.3 ± 1.6 


1.36 x 


0" 


-20 




3 


81.5 ±1.3 


3.07 x 


0" 


-11 




4 


80.5 ± 2.0 


1.35 x 


0" 


-10 




5 


83.5 ± 2.2 


1.30 x 


0" 


-12 




Avg. 


82.5 ± 6.4 


5.64 x 


0" 


-8 


5 


1 


82.7 ± 1.3 


6.55 x 


0" 


-12 




2 


86.3 ± 1 .3 


6.56 x 


0" 


-15 




3 


78.9 ± 1 .4 


2.17 x 


0" 


-9 




4 


82.2 ± 1.5 


6.55 x 


0" 


-12 




5 


81.0 ± 1.3 


1.35 x 


0" 


-10 




Avg. 


82.2 ± 2.7 


4.30 x 


0" 


-10 



Offline performances represented as classification accuracies estimated using 10 
runs of stratified 1 0-fold CV. The classification accuracy is defined as the 
probability of correctly classifying a trial given the feature, f*, i.e. 
P(correct \f*) = P{X \f* e X)P{X) + P(W \f* e W)P(W), where P(X \f* e X) and 
P(W \ f* e W) are defined in Online calibration in the Methods section, and P(X) 
and P(W) are the prior probabilities of idling and walking class, respectively. 



King etal. Journal of NeuroEngineering and Rehabilitation 2013, 10:77 
http://www.jneuroengrehab.eom/content/10/1/77 



Page 5 of 14 



[9]. Using the decoding model developed during the 
training procedure, the BCI system was set to run in 
the online mode while participants were prompted to 
alternate between performing idling and walking KMI for 
~2 min. The posterior probabilities were recorded and 
their histograms were plotted to determine two thresh- 
olds, one to initiate the ambulation of the avatar, Tyy, 
and the other to stop the avatar, Tx* These two thresh- 
olds were initially set as Tyy = median [P(W \ f* e W)} 
and Tx = median [P(W \f* el)}. Then, a brief online 



test was performed and based on the participants feed- 
back, the thresholds were adjusted as necessary. During 
online BCI operation, the posterior probabilities P(W |/*) 
corresponding to the most recent 1.5 sec of EEG data 
were averaged and compared to the thresholds Ty\? and 
Tx* and the state transitions were executed based on 
the binary state machine diagram given in Figure 1. 
This step was implemented in order to smooth the 
sequence of posterior probabilities and reduce noisy state 
transitions. 
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Figure 3 Feature Extraction Images of Participant 2. Feature extraction images of Participant 2 for all experimental sessions. Dark colors (red and 
blue) represent the areas that were responsible for encoding the differences between idling and walking KMI. The EEG power in the 9-1 3 Hz bins in 
the mid-frontal (FCz), central (Cz) and central-parietal (CPz) areas were the most salient. 
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BCI and virtual reality environment integration 

Using a simulated physics environment (Half-Life 2 
Garry's Mod, Valve Corporation, Bellevue, WA), a VRE 
was constructed to represent a flat grassland with 10 non- 
player characters (NPCs) arranged in a linear path in front 
of an avatar (see Figure 2). During online evaluation, the 
participants were assigned the task of utilizing idling and 
walking KMI to control the avatar s standing and walking, 
similar to [8,9]. A virtual joystick program [22] was used 



to interface the BCI software (MATLAB) and the VRE to 
enable the BCIs control of the avatar s walking. Full details 
on this integration process can be found in Wang et al. [9] . 

Online performance and assessment 

To assess the online BCI performance, participants were 
tasked to move the avatar within two body lengths of each 
NPC and remain idle at each location for at least 2 sec. On 
each experimental day, participants repeated this task for 
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Figure 4 Feature Extraction Images of Participant 5. Feature extraction images of Participant 5 for all experimental sessions. The EEG power in 
the 9-1 3 Hz bins in the mid-frontal (Fz), lateral central (C3 and C4) and lateral central-parietal (CP3 and CP4) areas were the most informative for 
encoding the differences between idling and walking KMI. 
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2-8 online sessions. This depended upon the participant s 
willingness and availability. In total, each participant 
underwent between 19 and 29 online sessions performed 
over 5 experimental days. Depending on the partici- 
pants availability, each experimental day was separated 
by approximately one week. Two performance measures 
were recorded for each session: the number of success- 
ful stops at the NPCs and the time taken to complete 
the course. The successful stops were defined as follows. 
Participants were given one point for dwelling at the desig- 
nated stop for at least 2 sec. Only a fraction of a point was 
given for dwelling at the designated stop between 0.5 sec 
and 2 sec (increasing proportionally with the dwell time 
duration). Also, no point was given for dwelling less than 
0.5 sec, and a full point was awarded for dwelling longer 
than 2 sec, but this inherently increased the time to com- 
pletion. Thus, the maximum successful stop score was 10 
points. In addition, participants were given a 20-min time 
limit to complete the course. If the participant could not 
finish the course within 20 min, the trial was interrupted 
and the number of successful stops achieved thus far was 
recorded. 

Control experiments 

The online performances (number of successful stops 
and completion time) were compared to random walk to 
determine if purposeful control was attained. The random 
walk performance was simulated by uniformly sampling 
the posterior probabilities between 0 and 1 and following 
the state transition rules with threshold values presented 
in Online calibration of the Methods section (details can 
be found in [9]). To ensure statistical significance, 1000 
Monte Carlo (MC) simulation runs were performed and 
the number of successful stops and completion times were 
logged (the 20-min completion time limit was enforced 



as above). In addition, the same task was performed by 
an able-bodied participant using a manually controlled 
physical joystick [9]. 

Statistical tests 

Purposeful control was defined as the ability to complete 
an online session within the allotted 20 min, with per- 
formances significantly different from random walk (p < 
0.01). This was ascertained by comparing the participants' 
online performances to MC simulations using a multi- 
variate analysis. To this end, the 2D probability density 
function (PDF) of each participants simulated random 
walk performance (consisting of two variables: number 
of successful stops and completion time) was estimated 
using the Parzen-Rosenblatt window method [18,23]. A 
constant- value PDF contour was then drawn through each 
participant s observed performance point and the corre- 
sponding p-value was determined as the fraction of the 
volume under the random walk PDF outside of this con- 
tour. These p-values were used to ascertain a statistically 
significant difference between the online performances 
and the simulated random walk. Finally, participants are 
deemed to have attained purposeful control once they 
have a single online session that is purposeful based on the 
definition above. 

Composite score 

Given the difficulty of interpreting multivariate perfor- 
mance measures across participants and sessions, a single 
composite score was defined as the following geometric 
mean: 

c = Jc^ct (3) 

where c s and c t are the normalized performance measures 
for the number of successful stops and completion times, 
respectively, i.e. 
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Figure 5 Calibration Histograms. Histograms of the posterior probability of walking KMI given idling, P(W \ f* e X), and walking KMI given 
walking, P(W \f* e W), for Participant 3. Note that P(W \f* e T) = 1 - P(X \f* e X). Dashed lines indicate the 25%, 50%, and 75% quantiles. 
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s 



^max ^min 

Here, s is the participants number of successful stops, 
Smax = 10 is the maximum number of successful stops, 
t is the participants completion time, £ max = 1200 sec 
is the maximum allowed time, and t m [ n = 201.52 sec is 
the minimum time required to complete the course while 
achieving 10 successful stops. The values of c s and c u and 
consequently c, range from 0 to 100%, where 100% cor- 
responds to a perfect performance. Note that the use of 
the geometric mean favors a performance that is balanced 
over a performance that sacrifices one performance mea- 
sure over the other (e.g. when a participant finishes the 
course in a short time while failing to make stops). Also 
note that the normalization of c s and c t ensures that the 
performance measures are unitless. 

Table 3 Online parameters 



Online parameters 



Participant 


Day 


Ti 


T w 


1 


1 


0.37 


0.47 




2 


0.42 


0.67 




3 


0.45 


0.55 




4 


0.22 


0.39 




5 


0.35 


0.44 


2 


1 


0.5 


0.5 




2 


0.07 


0.09 




3 


0.42 


0.45 




4 


0.6 


0.7 




5 


0.6 


0.66 


3 


1 


0.2 


0.3 




2 


0.57 


0.62 




3 


0.26 


0.75 




4 


0.7 


0.9 




5 


0.4 


0.9 


4 


1 


0.62 


0.8 




2 


0.61 


0.8 




3 


0.38 


0.45 




4 


0.4 


0.6 




5 


0.58 


0.65 


5 


1 


0.3 


0.85 




2 


0.4 


0.7 




3 


0.3 


0.7 




4 


0.4 


0.8 




5 


0.3 


0.85 



The stop (and start) parameters, Tx (and Tyy), for online operation as determined 
by the calibration session. 



Results 

Offline performances 

After participants underwent the training data collec- 
tion for each experimental day (see Training procedure 
section), participant-specific EEG decoding models were 
generated as described in the Decoding model genera- 
tion section. Cross-validation of these models resulted in 
offline classification accuracies that ranged from 60.5% 
to 92.3% (Table 2), with corresponding p-values between 
0.018 and 10 -20 , thus indicating that the classifier per- 
formances were well above the chance level (50%). In 
addition, the participants' average offline performances 
over 5 experimental days were 82.3%, 71.8%, 82.3%, 82.5%, 
and 82.2%, respectively. Finally, the overall average for all 
5 participants across all 5 experimental days was 80.2% ± 
8.62 (n = 25). 

Spatio-spectral feature extraction maps generated from 
offline analysis, as described in the Decoding model 
generation section, revealed salient brain areas and fre- 
quency bands underlying idling and walking KMI for each 
participant (see the supplementary material [Additional 
file 1] for all feature extraction images from 8-14 Hz). 
Qualitative analysis of these topographic maps revealed 
significant variations in the brain areas utilized by the 
participants while performing this task. For example, Par- 
ticipant 2 used mostly the Cz area (Figure 3), whereas Par- 
ticipant 5 used areas C3 and C4 (Figure 4). Furthermore, 
the utilized brain areas and frequency bands evolved for 
each participant over the 5 experimental days. For exam- 
ple, Participant 2 had growing activation of mid-frontal 
areas up to experimental day 4, followed by a slight shrink- 
age on day 5. However, one consistent feature among all 
participants was the activation of mid-frontal areas (over 
Cz or FCz and adjacent electrodes) mostly in the fi (8-12 
Hz) and low p (12-16 Hz) bands. 

Online calibration 

After the short calibration procedure (see Online calibra- 
tion in the Methods section), the state transition thresh- 
olds, Tx and Tyy, were determined from the distributions 
of the posterior probabilities, P(W \f*) (see Figure 5 for an 
example), and their values are presented in Table 3. Note 
that in an ideal situation, P(W \f* e W) and P(W \f* e 
T) should cluster around 1 and 0, respectively, but as long 
as these probabilities are separable, online BCI control 
should be achievable. The values of T% and Tyy ranged 
from 0.07 to 0.70 and from 0.09 to 0.90, respectively, and 
the average across all participants on all experimental days 
was 0.40 and 0.62 for Tx and Tyy, respectively. 

Online performances 

All participants were able to achieve purposeful online 
control immediately (on day 1), with the exception of Par- 
ticipant 2 (on day 2). The online BCI performances upon 
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achieving purposeful control are summarized in Table 4. 
The average stop score was 7.4 ± 2.3, and the average 
completion time was 273 ±51 sec across all online ses- 
sions (n = 124). Note that ideally, the course should be 
completed within 211 sec with a fixed walking time of 
191 sec and idling time of 20 sec, so the additional time 
spent to complete the course was due to errors (e.g. false 
stops, difficulties transitioning to the walk state, overes- 
timating the 2 sec dwelling time at each character). Also, 
only minimal lapses in BCI control occurred, as only 5 out 
of 124 online sessions (~4%) were non-purposeful. Exam- 
ples of the best online performances for each participant 
are shown in Figure 6 along with the simulated random 
walk PDFs. For additional comparison, the average perfor- 
mance achieved by a manually controlled physical joystick 
was 9.38 =b 0.95, and on their best day, Participants 3, 
4, and 5 achieved a similar number of successful stops; 
however, no participants were able to complete the course 
as fast as manual control. Two time-space courses of the 
best online sessions (see Figure 7) show only a single false 
alarm and a single omission for Participant 3, and no false 
alarms and only two omissions for Participant 5. Note that 
the online performances of the other subjects were not 
recorded with this level of detail (the computer code was 
modified in the late stage of the study to accommodate 



for this function), and it is likely that other subjects' false 
alarms and omissions were higher than the those reported 
in Figure 7. Therefore, while not universally applicable, 
these results illustrate the level of control achievable by 
this BCI system. Finally, a video showing a representa- 
tive online session from Participant 3 is provided as a 
supplementary material [see Additional file 2]. 

The composite scores were calculated from the online 
performance measures and are summarized in Table 5. 
In general, the performances improved significantly over 
time; the average on day 1 was 77.8% and the average on 
day 5 was 85.7% (p = 0.0302). For comparison, the com- 
posite score of the joystick task is also shown. On their 
best days, Participants 3, 4 and 5 achieved performances 
similar to those of the manually controlled joystick, reach- 
ing nearly perfect performances (100%). 

Discussion 

The results of this study show that participants with 
paraplegia or tetraplegia due to SCI can operate a non- 
invasive BCI-controlled avatar within a VRE to accom- 
plish a goal-oriented ambulation task. All participants 
gained purposeful online BCI control on the first day 
after undergoing a 10-min training session, with the 
exception of Participant 2, who did not attain control 



Table 4 Online performances 



Participant 




Completion time 


Successful stops 


Session 






mean ± std (sec) 


mean ± std 


breakdown 


1 


n=29 


275 ± 45 


6.2 ± 1 .8 


(26, 3) 




Best: Day 5 


298 ± 77 


6.8 ± 2.3 






Random Walk 


227 ±7 


5.0 ± 1.3 




2 


n=25 


271 ±66 


5.7 ±2.3 


(24,1) 




Best: Day 5 


293 ± 26 


8.1 ±1.2 






Random Walk 


1445 ±118 


9.0 ± 0.9 




3 


n=24 


277 ± 65 


9.4 ± 1 .3 


(24, 0) 




Best: Day 4 


231 ±8 


10.0 ±0.0 






Random Walk 


>1200 


0.1 ± 0.25 




4 


n=19 


289 ± 43 


8.3 ± 1 .8 


(18,1) 




Best: Day 1 


264 ± 12 


8.9 ± 0.3 






Random Walk 


>1200 


4.3 ± 0.7 




5 


n=27 


258 ±31 


7.7 ±2.1 


(27, 0) 




Best: Day 4 


260 ± 1 7 


10.0 ±0.0 






Random Walk 


>1200 


5.1 ±1.4 




All participants 




273 ±51 


7.4 ± 2.3 


(119, 5) 


Physical joystick 




205.72 ± 4.2 


9.38 ± 0.95 





Average online performances, including completion time, and number of successful stops for each participant, upon achieving purposeful control, which was on day 
2 for Participant 2, and on day 1 for all other participants. Note that the completion time consists of a fixed walking time (1 91 sec) and a variable amount of idling time. 
Also presented are the average online performances of each participant's best experimental day. Performances of random walk are shown for comparison. Sessions 
are also broken down by number of purposeful and non-purposeful performances, i.e. (p < 0.01 , p > 0.01 ). 
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until day 2. In addition, BCI control was maintained 
over the course of the study, and performance con- 
tinued to improve in 4 out of 5 participants. These 
findings suggest that a BCI-controlled lower extremity 
prosthesis for either gait rehabilitation or restoration may 
be feasible. 

Offline performances 

The offline classification accuracies varied across par- 
ticipants and experimental days, but were significantly 
above the chance level performance (50%). Similar to able- 
bodied participants engaged in the same task [8,9], a 
short 10-min training session was sufficient for the data- 
driven machine learning algorithm to generate accurate 
participant-specific EEG decoding models for this popula- 
tion. The topographic maps of these models (e.g. Figure 3 
and Figure 4) showed that the spatio-spectral features 
underlying the differences between walking and idling 
KMIs varied across participants and evolved over experi- 
mental days. The differences in the brain areas and EEG 
frequencies across participants may be due to variations 
in cortical reorganization following SCI [10-13], or due 
to differences in imageries employed by each participant 
(e.g. the KMI of walking instructions may have been inter- 
preted differently by each participant). Nevertheless, all 
participants showed activation of mid-frontal areas, which 
likely overlay the pre-motor and supplementary motor 
areas, as well as the pre-frontal cortex. Their activation 



during walking KMI is consistent with functional imaging 
findings, such as those in [24]. Another common pat- 
tern was the presence of activity near bilateral, lateral 
central-parietal electrodes, which likely represents the 
arm sensorimotor areas. A similar pattern was observed 
in able-bodied individuals [9], and is hypothesized to orig- 
inate from arm swing imagery. Finally, the evolution of the 
feature extraction maps over the 5 experimental days may 
be indicative of a neuro-plasticity process associated with 
practice and learning [25,26]. 

The spatio-temporal variations of walking KMI activa- 
tion patterns demonstrate the necessity of a data-driven 
machine learning approach for rapid acquisition of intu- 
itive BCI control. First, our approach accommodates for 
the variations of these activity patterns across partici- 
pants, as well as their evolution over time. Second, it facil- 
itates rapid acquisition of online BCI control, presumably 
by enabling participants to utilize intuitive mental strate- 
gies. The user training time necessary to acquire purpose- 
ful BCI control in this study is significantly shorter than 
those of other BCI studies where users must learn a com- 
pletely new cognitive skill to modulate pre-selected EEG 
features, such as the /x-rhythm over lateral central areas 
[27]. Finally, this approach carries a significant poten- 
tial value in the future practical implementation of BCI- 
prostheses, as it may drastically reduce the training time 
needed to attain purposeful and useful control of self- 
paced BCIs from a timescale of weeks to months to one of 
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Figure 6 Best Online Performances. Best online performances (completion times and stop scores) of each participant. Performances are marked 
by crosses with associated composite scores shown in parenthesis. PDF of the random walk simulations (contour lines) are also shown for Participant 
1, but are absent for other participants as the contours lie outside of the alloted 20-min limit. Note that all performances are purposeful (p < 0.01). 
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minutes to days. This in turn may significantly reduce the 
cost of training users to operate future BCI-prostheses. 

Online performances 

The results presented in Table 4 show that once pur- 
poseful control was achieved, it was maintained in 96% 
of all online sessions. In addition, 3 out of 5 participants 
achieved successful stop scores similar to those obtained 
using a manually controlled joystick. Even though no 
participants were able to complete the course as fast as 
manual control, it is encouraging that the average com- 
posite scores increased significantly over the course of 
the study. Furthermore, for Participants 3 and 5, the aver- 
age composite scores (Table 5) approached 100% by the 
toward of the study. Therefore, not only was online con- 
trol significantly different from random walk, but it was 
also meaningful. Given this trend, additional training and 



practice may help further improve performance, possibly 
to the point of approaching that of the manually controlled 
joystick. 

Comparison with other studies 

The similarities and differences between the present study 
and other related studies in the field are shown in Table 6. 
While this study represents a continuation of our prior 
work [8,9], it demonstrates the first successful appli- 
cation of a BCI-controlled walking simulator in a SCI 
population. The present approach also uses KMI of walk- 
ing/idling as a control strategy, which is intuitive because 
it matches the walking task. In contrast, related studies 
have used idling/foot imagery [28] or foot/hand imagery 
[29] to achieve control within a VRE. Furthermore, the 
data-driven machine learning approach to generate EEG 
decoding models in this study resulted in a very short 




Time (sec) 

Figure 7 Online BCI Operation Time-Space Courses. Two representative time courses of the best online sessions for (top) Participant 3 and 
(bottom) Participant 5, where both subjects were able to achieve a high number of successful stops with a short completion time. False positives 
(i.e. the avatar walked when the participant intended to stop) are marked by orange segments, and false negatives (i.e. the avatar stopped when the 
participant intended to walk) are marked by red segments. 
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Table 5 Composite online performance score 











Composite score (%) 






Participant 




Dayl 


Day 2 


Day 3 


Day 4 


Day 5 


1 




66.2 ±3.1 


76.8 ± 9.1 


75.0 ± 11.3 


74.4 ± 1 1 .6 


76.9 ± 1 1 .9 




Best 


69.7 


90 


85.6 


80.5 


89.9 


2 






69.9 ± 11.2 


69.3 ± 13.5 


59.9 ± 12.3 


85.4 ±5.4 




Best 




86.8 


89.3 


68.9 


90.4 


3 




86.4 ± 3.2 


87.5 ± 14.2 


93.0 ± 5.6 


98.4 ± 0.4 


97.3 ± 2.2 




Best 


89.9 


95.2 


98.5 


99 


98.6 


4 




91.5 ±1.7 


90.1 ±5.3 


89.7 ±4.1 


79.4 ± 16.3 


80.7 ± 2.3 




Best 


93.2 


95.5 


93.1 


96.1 


82.3 


5 




66.4 ± 9.0 


80.7 ±5.1 


93.5 ± 3.5 


96.9 ± 0.9 


88.7 ±6.1 




Best 


76.7 


86.8 


97.3 


98.2 


98.3 


Average 




77.8 ± 13.0 


79.8 ± 11.5 


81.7 ± 13.9 


80.7 ± 17.7 


85.7 ± 10.2 


Joystick 








96.5 ± 3.8 







Average and best composite online performance score for all participants for each experimental day as calculated using Eqs. 3 and 4. The composite scores for 
random walk and the physical joystick are also shown for comparison. Note that Participant 2 did not achieve purposeful control on day 1 and was therefore unable to 
participate in online sessions on this day. 



training time, while those presented in [28,29] resulted in 
a significantly longer training time (see Table 6). Finally, 
both our prior work [9] and [28] report on the BCI 
performances of a single SCI participant, whereas the cur- 
rent study includes a larger SCI population. This suggests 
that our BCI-VRE system may be generalizable in SCI 
individuals. 

Future work 

The success of this study suggests that restoring BCI- 
controlled ambulation after SCI may be feasible. To for- 
mally test this concept, integration of our BCI system with 
a physical ambulation prosthesis, such as a robotic gait 
orthosis (e.g. Lokomat, Hocoma, Volketswil, Switzerland) 
or an FES system (e.g. Parastep, Sigmedics, Fairborn, OH) 
is necessary. If such a system is successfully tested in a 
small SCI population, clinical trials to develop a com- 
mercial BCI-prosthesis would be warranted. Finally, these 
research efforts may ultimately lead to the development of 
a permanent BCI prosthesis for ambulation; for example, 
the integration of an invasive BCI system with an inva- 
sive FES system for ambulation. This type of system would 



eliminate the need for any preparation or mounting of 
EEG, while potentially achieving better performances. 

Conclusions 

This study shows that SCI participants can purposefully 
operate a self-paced BCI-VRE system in real time, and 
that the current BCI design approach is able to over- 
come the potential problems associated with variations in 
neurophysiology due to cortical reorganization after SCI, 
learning and plasticity processes, and differences in KMI 
strategies. Furthermore, the system satisfies the require- 
ments of an ideal BCI-lower extremity prosthesis [9], 
namely: intuitiveness, robustness, and short training time. 
The operation of the system is intuitive as it enabled par- 
ticipants to use walking KMI to control the ambulation 
of the avatar. The system is robust in that the data-driven 
decoding methodology successfully accommodated for 
person-to-person and day-to-day variations in the neu- 
rophysiological underpinnings of idling and walking KMI 
behaviors. In addition, participants were able to main- 
tain purposeful online control over the course of several 
weeks, further underscoring the systems robustness over 



Table 6 Comparison with other studies 


Study 


Mental strategy 


Training time 


Sample size 


Present 


KMI of walking/idling 


10 min 


5 SCI 


Wang etal. [9] 


KMI of walking/idling 


10 min 


8 AB, 1 SCI 


Wang etal. [8] 


KMI of walking/idling 


12 min 


3 AB 


Pfurtschelleretal. [29] 


Foot/hand motor imagery 


3-5 mo 


3 AB 


Leeb et al. [28] 


Foot/idling motor imagery 


4 mo 


1 SCI 



A comparison of the present study with similar studies in the field. 
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time. Finally, the system required only a short training 
time, as BCI control was generally attained after only a 
10-min training data collection procedure followed by a 
2-min calibration session on the 1 st experimental day (for 
4 out of the 5 participants). 

In summary, the high level of control achieved by SCI 
subjects over the course of the study indicates that BCI- 
controlled lower extremity prostheses for gait rehabilita- 
tion or restoration after SCI may be feasible. The proposed 
BCI-VRE system may also serve as a training platform 
for BCI-controlled lower extremity prostheses once they 
become widely available. 



Additional files 



Additional file 1 : Feature extraction images of all participants for all 
experimental sessions over the 8-1 4 Hz frequency band. Dark colors 
(red and blue) represent the areas that were responsible for encoding the 
differences between idling and walking KMI. 

Additional file 2: Subject with tetraplegia due to syringomyelia uses 
the BCI to control the ambulation of a virtual reality avatar. To move 
the avatar, the subject imagines walking, and to stop, the subject relaxes. 
The goal is to move the avatar across the field and make 1 0 stops within 2 
body lengths of non-player characters. Each stop should be at least 2 
seconds long. This subject can complete the task while making all 1 0 stops 
with a completion time that is only 15 seconds longer than that of a 
manually controlled joystick. 
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